Decentralized Federated Learning With Intermediate Results in Mobile Edge Computing

The emerging Federated Learning (FL) permits all workers (e.g., mobile devices) to cooperatively train a model using their local data at the network edge. In order to avoid the possible bottleneck of conventional parameter server architecture, the decentralized federated learning (DFL) is developed on the peer-to-peer (P2P) communication. In DFL, model exchanging among workers is usually regarded as an atomic operation, which largely affects the total bandwidth consumption during model training. Given the limited communication resource on workers, model exchanging will pose a great challenge when meeting with the large-scale models. Herein, we propose to let workers exchange the
intermediate results
, instead of the entire model, with each other. We provide theoretical analysis of DFL based on intermediate result exchanging, which reveals the relationship between the training performance and the exchanging interval (i.e., the number of local updating iterations) of intermediate results. According to the convergence bound, we propose an adaptive exchanging interval (or frequency) algorithm called Fed-IR, which optimizes the trade-off between communication cost and training performance. Extensive simulation results show that compared with the model exchanging methods, our proposed algorithms can save communication traffic of around 42%
$\sim$
81% while still achieving the similar accuracy.